Mobile apparatus for executing sensing flow for mobile context monitoring, method of executing sensing flow using the same, method of context monitoring using the same and context monitoring system including the same

ABSTRACT

A mobile apparatus includes a sensing handler and a processing handler. The sensing handler includes a plurality of sensing operators. The sensing operator senses data during a sensing time corresponding to a size of C-FRAME and stops sensing during a skip time. The C-FRAME is a sequence of the sensed data to produce a context monitoring result. The processing handler includes a plurality of processing operators. The processing operator executes the sensed data of the sensing operator in a unit of F-FRAME. The F-FRAME is a sequence of the sensed data to execute a feature extraction operation.

PRIORITY STATEMENT

This application claims priority under 35 U.S.C. §119 to Korean PatentApplication No. 10-2013-0121233, filed on Oct. 11, 2013 in the KoreanIntellectual Property Office (KIPO), the contents of which are hereinincorporated by reference in their entireties.

BACKGROUND

1. Technical Field

Example embodiments relate to a mobile apparatus, a method of executinga sensing flow using the mobile apparatus, a method of contextmonitoring using the mobile apparatus and a context monitoring systemhaving the mobile apparatus. More particularly, example embodimentsrelate to a mobile apparatus capable of improving utility of applicationby coordinating resource uses of concurrent executing applications, amethod of executing a sensing flow using the mobile apparatus, a methodof context monitoring using the mobile apparatus and a contextmonitoring system having the mobile apparatus.

2. Description of the Related Art

Recent paradigm of information communication technology may be aubiquitous computing, a ubiquitous network, a pervasive computing and soon. “Ubiquitous” means that a user may easily get any desiredinformation anytime and anywhere. In an upcoming ubiquitous age, smartobjects, having computing and communication function, may recognize adynamic environment and be adaptive to the dynamic environment. In otherwords, the smart objects may have a context awareness feature.

A personal area network (PAN) is one of the core technologies realizingthe ubiquitous network having the context awareness feature. The PAN isa network which is provided to a person to communicate in a close range.The person using the PAN may be connected with various devices in about10 m with respect to the person.

The PAN is suitable for a context monitoring application, which providesproper services in response to an action of the user, a status of theuser and an environment around the user. In the PAN environment, thenetwork is operated around the person so that a portable mobileapparatus, capable of receiving data from various sensors and outputtingcontext information to the context monitoring applications, may be acore platform. For example, a mobile terminal may recognize a context ofthe user by collecting and transmitting the sensed data, and may provideinformation to the context monitoring application by analyzing thesensed data. The context monitoring application may provide properservices to the user according to the context of the user. Accordingly,the mobile apparatus capable of supporting a number of the contextmonitoring applications may be necessary.

Emerging continuous mobile sensing applications may significantly changeworkload patterns imposed on the mobile apparatus. Going beyond theconfines of typical user-interactive mobile applications such as webbrowsers and games, the user-interactive mobile applicationscontinuously run in the background and provide autonomous,situation-aware services without a user's intervention. Such concurrentworkloads will raise an unprecedented challenge, incurring severeresource contention on the resource-scarce mobile apparatus. Thecontention is aggravated due to the continuous heavy CPU consumption ofindividual sensing applications to process high-rate sensor data. Moreimportant, such sensing and processing workloads may be handled nearreal-time to provide timely services. Even worse, the total resourceavailability might be limited further, deteriorating the contention;user's may not exhaust the whole CPU cycles and battery only forbackground applications. Under such contentious situation, greedyresource use by an application may result in serious degradation ofservice qualities of the other applications. The performance of otherdaily use of the mobile apparatus may be also degraded.

SUMMARY

Example embodiments provide a mobile apparatus improving utility ofapplication by coordinating resource uses of concurrent executingapplications.

Example embodiments also provide a method of executing a sensing flowusing the mobile apparatus.

Example embodiments also provide a method of monitoring a context usingthe mobile apparatus.

Example embodiments also provide a context monitoring system having themobile apparatus.

In an example mobile apparatus according to the present inventiveconcept, the mobile apparatus includes a sensing handler and aprocessing handler. The sensing handler includes a plurality of sensingoperators. The sensing operator senses data during a sensing timecorresponding to a size of C-FRAME and stops sensing during a skip time.The C-FRAME is a sequence of the sensed data to produce a contextmonitoring result. The processing handler includes a plurality ofprocessing operators. The processing operator executes the sensed dataof the sensing operator in a unit of F-FRAME. The F-FRAME is a sequenceof the sensed data to execute a feature extraction operation.

In an example embodiment, the C-FRAME may include a plurality of theF-FRAMEs.

In an example embodiment, the mobile apparatus may further include aflow analyzer receiving information for a sensing flow from anapplication and determining a size of the C-FRAME of the sensing flowand a size of the F-FRAME of the sensing flow.

In an example embodiment, the flow analyzer may receive a necessarymonitoring interval and a monitoring delay from the application. Thenecessary monitoring interval may represent how often the applicationneeds to monitor a user's situation. The monitoring delay may representtime taken to generate the context monitoring result from a moment thata final F-FRAME in the C-FRAME is ready.

In an example embodiment, the mobile apparatus may further include aflow execution planner determining a monitoring interval of the C-FRAMEbased on the size of the C-FRAME, the size of the F-FRAME and thenecessary monitoring interval and outputting the monitoring interval ofthe C-FRAME to the sensing handler. The monitoring interval of theC-FRAME of the sensing flow may be substantially equal to a sum of thesensing time and the skip time.

In an example embodiment, the mobile apparatus may further include aresource monitor determining a CPU availability of the mobile apparatusand outputting the CPU availability to the flow execution planner. Theflow execution planner may determine the monitoring interval of theC-FRAME of the sensing flow based on the size of the C-FRAME, the sizeof the F-FRAME, the necessary monitoring interval and the CPUavailability.

In an example embodiment, the necessary monitoring interval may have atype of utility function, the utility function having utility values ofthe application according to the monitoring intervals.

In an example embodiment, when the monitoring interval of a plurality ofsensing flows is pi and the utility function of the sensing flows is ui,the flow execution planner may determine the monitoring interval as aformula, maximize Σui(pi).

In an example embodiment, when cti is a CPU time required to process theC-FRAME of a flow, FLOWi and 1−CPUf is a CPU availability for theapplications, the flow execution planner may determine the monitoringinterval under a constraint of Σcti/pi≦1−CPUf.

In an example embodiment, when eci is energy required to process theC-FRAME of a flow, FLOWi and Elimit is an energy availability for theapplications, the flow execution planner may determine the monitoringinterval under a constraint of Σeci/pi≦Elimit.

In an example embodiment, the sensing flows may include respectiveF-FRAME queues. The mobile apparatus may further include a flowscheduler determining an execution order of the F-FRAMEs by selecting aF-FRAME queue among the F-FRAME queues.

In an example embodiment, the flow scheduler may determine the executionorder of the F-FRAMEs based on the monitoring delay.

In an example embodiment, a function of satisfy(ci) represents 1 if acontext monitoring result for i-th flow, FLOWi is generated byprocessing the F-FRAMEs in the F-FRAME queue in the monitoring delay andrepresents 0. The flow scheduler determines the execution order of theF-FRAMEs using a formula, max imizeΣsatisfy(ci).

In an example embodiment, a j-th F-FRAME of an i-th C-FRAME (C-FRAMEi)is F-FRAMEi,j, di is a tolerable delay of the FLOWi, ri,j is a remainingtime to collect the remaining F-FRAMEs in the C-FRAMEi, epti,j is anexpected time to process the unprocessed F-FRAMEs in the C-FRAMEi and aslack time of the F-FRAMEi,j is determined as a formula,slacktime(F-FRAMEi,j)=di+ri, j-epti, j. The flow scheduler may selectthe F-FRAME having a least slack time.

In an example embodiment, the mobile apparatus may further include a CPUscheduler outputting a CPU quota to the flow execution planner.

In an example method of executing a sensing flow according to thepresent inventive concept, the method includes sensing data during asensing time corresponding to a size of C-FRAME and stopping sensingduring a skip time using a plurality of sensing operators and executingthe sensed data of the sensing operator in a unit of F-FRAME using aprocessing operator. The C-FRAME is a sequence of the sensed data toproduce a context monitoring result. The F-FRAME is a sequence of thesensed data to execute a feature extraction operation.

In an example embodiment, the C-FRAME may include a plurality of theF-FRAMES.

In an example embodiment, the method may further include receivinginformation for a sensing flow from an application and determining asize of the C-FRAME of the sensing flow and a size of the F-FRAME of thesensing flow.

In an example embodiment, the method may further include receiving anecessary monitoring interval and a monitoring delay from theapplication. The necessary monitoring interval may represent how oftenthe application needs to monitor a user's situation. The monitoringdelay may represent time taken to generate the context monitoring resultfrom a moment that a final F-FRAME in the C-FRAME is ready.

In an example embodiment, the method may further include determining amonitoring interval of the C-FRAME based on the size of the C-FRAME, thesize of the F-FRAME and the necessary monitoring interval. Themonitoring interval of the C-FRAME of the sensing flow may besubstantially equal to a sum of the sensing time and the skip time.

In an example embodiment, the method may further include determining aCPU availability of the mobile apparatus. The monitoring interval of theC-FRAME of the sensing flow may be determined based on the size of theC-FRAME, the size of the F-FRAME, the necessary monitoring interval andthe CPU availability.

In an example embodiment, the necessary monitoring interval may have atype of utility function, the utility function having utility values ofthe application according to the monitoring intervals.

In an example embodiment, the sensing flows may include respectiveF-FRAME queues. The method may further include determining an executionorder of the F-FRAMEs by selecting a F-FRAME queue among the F-FRAMEqueues.

In an example embodiment, the execution order of the F-FRAMEs, may bedetermined based on the monitoring delay.

In an example method of monitoring a context according to the presentinventive concept, the method includes receiving a context monitoringrequest from an application, sensing data during a sensing timecorresponding to a size of C-FRAME and stopping sensing during a skiptime using a plurality of sensing operators based on the contextmonitoring request, executing the sensed data of the sensing operator ina unit of F-FRAME using a processing operator based on the contextmonitoring request and outputting the context monitoring result to theapplication. The C-FRAME is a sequence of the sensed data to produce acontext monitoring result. The F-FRAME is a sequence of the sensed datato execute a feature extraction operation.

In an example context monitoring system according to the presentinventive concept, the context monitoring system includes a sensor, amobile apparatus and an application. The sensor generates sensed data.The mobile apparatus includes a sensing handler and a processinghandler. The sensing handler includes a plurality of sensing operators.The sensing operator senses data during a sensing time corresponding toa size of C-FRAME and stops sensing during a skip time. The C-FRAME is asequence of the sensed data to produce a context monitoring result. Theprocessing handler includes a plurality of processing operators. Theprocessing operator executes the sensed data of the sensing operator ina unit of F-FRAME. The F-FRAME is a sequence of the sensed data toexecute a feature extraction operation. The application receives thecontext monitoring result from the mobile apparatus.

According to the mobile apparatus, the method of executing the sensingflow, the method of monitoring the context, and the context monitoringsystem, contentious concurrent workloads of the applications may becoordinated on the whole so that potential imbalances in the servicequalities of applications may be effectively resolved. In addition,developer may easily build sensing applications without concerningsevere resource contention and the dynamics caused by concurrentapplications. Thus, the resource uses of the contentious concurrentapplications are coordinated so that the utility of the application maybe maximized.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other features and advantages of the present inventiveconcept will become more apparent by describing in detailed exampleembodiments thereof with reference to the accompanying drawings, inwhich:

FIG. 1 is a block diagram illustrating a context monitoring systemaccording to an example embodiment of the present inventive concept;

FIG. 2 is a detailed block diagram illustrating the context monitoringsystem of FIG. 1;

FIG. 3A is a conceptual diagram illustrating a sensing flow of a firstcontext monitoring application executed by the mobile apparatus of FIG.1;

FIG. 3B is a conceptual diagram illustrating a sensing flow of a secondcontext monitoring application executed by the mobile apparatus of FIG.1;

FIG. 3C is a conceptual diagram illustrating a sensing flow of a thirdcontext monitoring application executed by the mobile apparatus of FIG.1;

FIG. 4 is a conceptual diagram illustrating frame externalization of asensed data stream executed by the mobile apparatus of FIG. 1;

FIGS. 5A and 5B are conceptual diagrams illustrating C-FRAME basedresource allocation executed by the mobile apparatus of FIG. 1;

FIG. 6 is a conceptual diagram illustrating C-FRAME based flowcoordination executed by the mobile apparatus of FIG. 1;

FIG. 7 is a conceptual diagram illustrating an operation of a flowexecutor of FIG. 2; and

FIG. 8 is a conceptual diagram illustrating slack time estimation ofF-FRAME executed by a flow scheduler of FIG. 2.

DETAILED DESCRIPTION OF THE EMBODIMENTS

The present inventive concept now will be described more fullyhereinafter with reference to the accompanying drawings, in whichexemplary embodiments of the present invention are shown. The presentinventive concept may, however, be embodied in many different forms andshould not be construed as limited to the exemplary embodiments setfourth herein.

Rather, these exemplary embodiments are provided so that this disclosurewill be thorough and complete, and will fully convey the scope of thepresent invention to those skilled in the art. Like reference numeralsrefer to like elements throughout.

It will be understood that, although the terms first, second, third,etc. may be used herein to describe various elements, components,regions, layers and/or sections, these elements, components, regions,layers and/or sections should not be limited by these terms. These termsare only used to distinguish one element, component, region, layer orsection from another region, layer or section. Thus, a first element,component, region, layer or section discussed below could be termed asecond element, component, region, layer or section without departingfrom the teachings of the present invention.

The terminology used herein is for the purpose of describing particularexemplary embodiments only and is not intended to be limiting of thepresent invention. As used herein, the singular forms “a,” “an” and“the” are intended to include the plural forms as well, unless thecontext clearly indicates otherwise. It will be further understood thatthe terms “comprises” and/or “comprising,” when used in thisspecification, specify the presence of stated features, integers, steps,operations, elements, and/or components, but do not preclude thepresence or addition of one or more other features, integers, steps,operations, elements, components, and/or groups thereof.

Unless otherwise defined, all terms (including technical and scientificterms) used herein have the same meaning as commonly understood by oneof ordinary skill in the art to which this invention belongs. It will befurther understood that terms, such as those defined in commonly useddictionaries, should be interpreted as having a meaning that isconsistent with their meaning in the context of the relevant art andwill not be interpreted in an idealized or overly formal sense unlessexpressly so defined herein.

All methods described herein can be performed in a suitable order unlessotherwise indicated herein or otherwise clearly contradicted by context.The use of any and all examples, or exemplary language (e.g., “suchas”), is intended merely to better illustrate the invention and does notpose a limitation on the scope of the invention unless otherwiseclaimed. No language in the specification should be construed asindicating any non-claimed element as essential to the practice of theinventive concept as used herein.

Hereinafter, the present inventive concept will be explained in detailwith reference to the accompanying drawings.

FIG. 1 is a block diagram illustrating a context monitoring systemaccording to an example embodiment of the present inventive concept.

Referring to FIG. 1, the context monitoring system includes a mobileapparatus 100, an external sensor 200 and a context monitoringapplication 300.

The mobile apparatus 100 receives a context monitoring request (CMQ)from the context monitoring application 300 operating a contextmonitoring function. The mobile apparatus 100 receives sensed data SDfrom the external sensor 200. The mobile apparatus 100 determinescontext monitoring result CMR. The mobile apparatus 100 outputs thecontext monitoring result CMR to the application 300. A structure of themobile apparatus 100 is explained in detail referring to FIG. 2.

The external sensor 200 provides the sensed data SD to the mobileapparatus 100. Alternatively, the external sensor 200 may providefeature data extracted from the sensed data SD to the mobile apparatus100.

For example, the external sensor 200 may include a plurality of sensors.The external sensor 200 may be a light sensor, a temperature sensor, aposition sensor, a dust sensor, an ultraviolet ray sensor, a hygrometer,a carbon dioxide detector, an ambient sound detector, an accelerometerand so on. Accordingly, the external sensor 200 may detect light,temperature, position, dust quantity, ultraviolet ray quantity,humidity, carbon dioxide quantity, ambient sound, acceleration and soon. The external sensor 200 may be a wearable sensor attached to a humanbody who is a user of the mobile apparatus 100.

The sensed data SD from the external sensor 200 are provided to themobile apparatus 100, and are used to determine whether the sensed dataSD satisfies a context requested by the context monitoring application300.

The context monitoring application 300 requests the CMQ according to anobject of program to the mobile apparatus 100. The context monitoringapplication 300 receives the context monitoring result CMR from themobile apparatus 100.

FIG. 2 is a detailed block diagram illustrating the context monitoringsystem of FIG. 1.

Referring to FIGS. 1 and 2, the mobile apparatus 100 includes a flowanalyzer 110, a resource monitor 120, a flow scheduler 130, a flowexecution planner 140, a flow executor and a CPU scheduler 170. The flowexecutor includes a processing handler 150 and a sensing handler 160.The mobile apparatus may further include an internal sensor 180.

The flow analyzer 110 receives the context monitoring request CMQ fromthe context monitoring application 300. The context monitoring requestCMQ may include sensing flow information and an execution requirement.

The sensing flow information represents a flow of the sensinginformation which the context monitoring application 300 demands. Thesensing flow information may correspond to a data flow graph shown inFIGS. 3A to 3C.

For example, the sensing flow information may be a data flow programmingmodel of an XML interface. The sensing flow information of the XMLinterface may include an operator and an edge between the operators. Theoperator represents a unit of computation or I/O. The edge representsdata dependencies between the operators.

The execution requirement includes a necessary monitoring interval and amonitoring delay.

The necessary monitoring interval represents how often the application300 needs to monitor the user's situation. For example, CalorieMonapplication may require capturing the user's physical activity everyseveral seconds to compute the total calorie expenditure of a day.Typically, the shorter the interval is, the higher the utility of theapplication is. Applications 300 specify the preferred monitoringinterval tied with utility values for each interval, which is expressedas a utility function. The mobile apparatus 100 may coordinate theresource uses to maximize the utility of the context monitoringapplications 300.

The monitoring delay represents a maximum tolerable delay of the contextmonitoring applications 300. The delay means the time to deliver finalcontext monitoring results CMR to the application 300 from the moment ofsensing.

In addition, the execution requirement may further include an accuracyof the context monitoring result CMR.

The flow analyzer 110 determines a frame structure based on the sensingflow. The flow analyzer 110 may determine a size of C-FRAME and a sizeof F-FRAME based on the sensing flow. The flow analyzer 110 outputs thesize of the C-FRAME and the size of the F-FRAME to the flow executionplanner 140.

Generally, the sensed data stream is generated as a flat sequence ofdata. However, taken by the application 300, the sensed data stream isinterpreted with a tailored structure. The application 300 extracts anddetermines a specific context from the sensed data stream. Knowledge onthe frame structure of the sensed data stream gives useful hints inexecuting the application 300 and planning the resource use of theapplication 300, especially when a system deal with multiple concurrentapplications under a resource contentious circumstance.

When an application A takes a sensed data stream S as input, a virtualstructure of S may be extracted by inspecting the flow of A. The processof externalizing the virtual structure of S with respect to A is calledas framing or frame externalization. The resulting structural entity ofS is called as a frame. The sensed data stream may be framed differentlyby different applications 300. Also, the sensed data stream may beframed differently even for the same application 300.

The C-FRAME represents a context-frame. The C-FRAME means a sequence ofthe sensed data to produce the context monitoring result CMR. The mobileapparatus 100 uses the C-FRAME as a basic unit of data processing andresource allocation.

The F-FRAME represents a feature-frame. The F-FRAME means a sequence ofthe sensed data to execute a feature extraction operation. The C-FRAMEincludes a plurality of the F-FRAMEs.

Framing the sensed data stream is explained referring to FIG. 4 indetail.

The flow analyzer 110 outputs the monitoring delay of the executionrequirement to the flow scheduler 130. The flow analyzer 110 outputs thenecessary monitoring interval of the execution requirement to the flowexecution planner 140. For example, the necessary monitoring intervalmay have a type of a utility function representing utility valuesaccording to the monitoring intervals.

The resource monitor 120 receives profile data from the flow executor.The profile data may include a CPU usage and an energy usage of themobile apparatus 100. The CPU usage and the energy usage may be changedin real time according to other applications. The profile data mayinclude a CPU usage and an energy usage of the processing handler 150and the sensing handler 160.

The resource monitor 120 outputs the CPU usage and the energy usage tothe flow execution planner 140.

For example, the CPU usage may be determined using an elapsed time forthe C-FRAME execution, the time between the start and end of the C-FRAMEexecution.

For example, the energy usage may be determined using an offlineprofiling method. In the offline profiling method, the energy profilesfor sensors, CPU and network interfaces which are major hardwarecomponents used by the sensing application 300 are prebuilt. The energyusage of the C-FRAME is estimated by adding the energy consumed toperform the operations to collect and process the C-FRAME based on theenergy profiles.

The flow scheduler 130 receives the monitoring delay from the flowanalyzer 110. The flow scheduler 130 determines an execution order ofthe F-FRAMEs of the sensing handler 160 based on the monitoring delay.

A method of determining the execution order of the F-FRAMEs by the flowscheduler 130 is explained referring to FIG. 8.

The flow execution planner 140 receives the size of the C-FRAME, thesize of the F-FRAME and the necessary monitoring interval from the flowanalyzer 110. The flow execution planner 140 receives the CPU usage andthe energy usage from the resource monitor 120. The flow executionplanner 140 may further receive a CPU quota allowing the execution ofthe sensing application 300 from the CPU scheduler 170.

The flow execution planner 140 controls the sensing flow of the sensinghandler 160 using the size of the C-FRAME, the size of the F-FRAME andthe necessary monitoring interval.

The flow execution planner 140 determines a monitoring interval of asensing operator of the sensing handler 160 using the size of theC-FRAME, the size of the F-FRAME and the necessary monitoring interval.A skip time may be determined by the size of the C-FRAME and themonitoring interval. The sensing handler 160 senses the datacorresponding to the size of the C-FRAME, and stops sensing the dataduring the skip time. The sensing handler 160 repeats the sensing andthe suspension in a cycle of the monitoring interval.

When the sensing handler 160 includes a plurality of the sensingoperators, the flow execution planner 140 may determine the monitoringinterval for each sensing operator.

The flow execution planner 140 may control the sensing flow of thesensing handler 160 using the size of the C-FRAME, the size of theF-FRAME, the necessary monitoring interval, the CPU usage and the energyusage. The flow execution planner 140 may control the sensing flow ofthe sensing handler 160 further using the CPU quota. The CPU quota meansa limit of a CPU usage allowed to the context monitoring application300. The CPU quota may be set by an operating system (OS) or set by theuser.

When the concurrent sensing flows are FLOWi and the monitoring intervalsof the C-FRAMEs are pi, the flow execution planner 140 determines themonitoring interval pi of the C-FRAME using following Formulas 1 to 3.

maximizeΣui(pi)  [Formula 1]

Σcti/pi≦1−CPUf  [Formula 2]

Σeci/pi≦Elimit  [Formula 3]

In Formula 1, ui means a utility function of the FLOWi. The utilityfunction represents a utility value of the application according to themonitoring interval pi. The flow execution planner 140 determines themonitoring interval pi to maximize the utility of the plurality of thesensing flows.

In Formula 2, cti means a CPU time required to process the C-FRAME ofFLOWi. CPUf means the CPU portion taken by other applications. 1−CPUfmeans the CPU availability for the sensing applications. Alternatively,the CPU availability may be set by the user regardless of the CPUf whichmeans the CPU portion taken by other applications. The flow executionplanner 140 determines the monitoring interval pi under the constraintsof Formula 2.

In Formula 3, eci means energy required to process the C-FRAME of FLOWi.Elimit means an energy availability for the sensing applications. Theflow execution planner 140 determines the monitoring interval pi underthe constraints of Formula 3.

The CPU usage may be determined using eti which means elapsed time forthe C-FRAME execution of FLOWi, the time between the start and end ofthe C-FRAME execution. Using the elapsed time eti, cti in Formula 2 maybe approximated as eti*(1−CPUf). Accordingly, Formula 2 may berepresented as following Formula 4.

Σeti/pi≦1  [Formula 4]

A specific operation of the flow execution planner 140 may be explainedreferring to FIGS. 5 and 6 in detail.

The processing handler 150 includes a plurality of processing operators.The processing handler 150 controls an operation of the processingoperator. The processing handler 150 controls an operation of theprocessing operator according to scheduling of the flow scheduler 130.The processing operators may be operated in a unit of the F-FRAME.

The sensing handler 160 includes a plurality of sensing operators. Thesensing handler 160 controls an operation of the sensing operatoraccording to planning of the flow execution planner 140. The sensingoperators receive the monitoring interval information. The sensingoperators sense the data in a unit of C-FRAME and stop sensing duringthe skip time based on the monitoring interval information.

The operations of the processing handler 150 and the sensing handler 160are explained referring to FIG. 7 in detail.

The CPU scheduler 170 outputs the CPU quota to the flow executionplanner 140. The CPU scheduler 170 may be included in the operatingsystem.

The internal sensor 180 outputs internal sensed data ISD to the sensingoperator of the sensing handler 160. Alternatively, the internal sensor180 may output feature data extracted from the internal sensed data ISDto the sensing operator.

For example, the internal sensor 180 may include a plurality of internalsensors. The internal sensor 180 may be a light sensor, a temperaturesensor, a position sensor, a dust sensor, an ultraviolet ray sensor, ahygrometer, a carbon dioxide detector, an ambient sound detector, anaccelerometer and so on. Accordingly, the internal sensor 180 may detectlight, temperature, position, dust quantity, ultraviolet ray quantity,humidity, carbon dioxide quantity, ambient sound, acceleration and soon.

The internal sensed data ISD from the internal sensor 180 are providedto the sensing handler 160, and are used to determine whether theinternal sensed data ISD satisfies a context requested by the contextmonitoring application 300.

FIG. 3A is a conceptual diagram illustrating a sensing flow of a firstcontext monitoring application executed by the mobile apparatus 100 ofFIG. 1. FIG. 3B is a conceptual diagram illustrating a sensing flow of asecond context monitoring application executed by the mobile apparatus100 of FIG. 1. FIG. 3C is a conceptual diagram illustrating a sensingflow of a third context monitoring application executed by the mobileapparatus 100 of FIG. 1.

In FIGS. 3A to 3C, a rectangular box in a lowermost level represents anexternal sensor or an internal sensor. Chamfered rectangular boxesrepresent sensing operators and processing operators. Numbers betweenthe operators represent window sizes of sensed data streams.

Referring to FIGS. 3A, 3B and 3C, FIG. 3A represents a sensing flow of acontext monitoring application, ChildMon. ChildMon allows workingparents to be aware of realtime activities of their kindergarten childduring classes and filedtrips. Childmon monitors children's activitieslike talking and playing using a backpack-attached mobile apparatus andnotifies the parents of the distinguished activities. Childmon sensessound data to determine the children's activities.

In FIG. 3A, the sensing operator, SOUND, continuously samples audio dataat 8 kHz. The audio data is transmitted two processing operators RMS andFFT which are connected to SOUND. Results of RMS and FFT are furtherprocessed through a series of operators and transmitted to GMM (GaussianMixture Model) and SMOOTHING. The children's activity is determined bySMOOTHING

FIG. 3B represents a sensing flow of a context monitoring application,IndoorNavi. IndoorNavi helps a mobile user to navigate large-scalebuilding complexes. For the navigation, the application continuouslylocalizes a location of the mobile apparatus.

In FIG. 3B, the sensing operator, SOUND, continuously generates sounddata at 44.1 kHz and transmits the sound data to a processing operator,WINDOW FUNCTION next to SOUND. A result of WINDOW FUNCTION istransmitted to NEAREST-NEIGHBOR through a series of operators.

FIG. 3C represents a sensing flow of a context monitoring application,CalorieMon. CalorieMon monitors a user's physical activities duringdaily exercise and estimates real-time caloric expenditure of the user.The application utilizes a plurality of accelerometers disposed indifferent body portions.

In FIG. 3C, the sensing operators, ACCEL_X and ACCEL_Y, continuouslysamples acceleration data at 50 Hz and transmits the acceleration datato a processing operator, FFT, CORRELATION connected to ACCEL_X andACCEL_Y. Results of FFT, CORRELATION are transmitted to DECISION TREEthrough a series of operators. By DECISION TREE, the caloric expenditureof the user is calculated.

FIG. 4 is a conceptual diagram illustrating frame externalization of asensed data stream executed by the mobile apparatus of FIG. 1.

FIG. 4 represents that ChildMon frames the sound stream into the C-FRAMEand the F-FRAME.

Referring to FIGS. 1 to 4, the flow analyzer 110 may determine a size ofthe C-FRAME and a size of the F-FRAME based on the sensing flow ofChildMon.

In FIG. 4, the sound stream of 8 kHz is framed by ChildMon in threelayers. In a first layer, a sequence of 512 consecutive samples isframed for feature extraction. In a second layer, 20 of the first layerframes are combined for classification. In a third layer, 3 of thesecond layer frames are combined to generate a final context monitoringresult. Therefore, 512*20*3 samples generate the final contextmonitoring result. Herein, the size of the F-FRAME is 512 samples. Thesize of the C-FRAME is 512*20*3 samples.

FIGS. 5A and 5B are conceptual diagrams illustrating C-FRAME basedresource allocation executed by the mobile apparatus 100 of FIG. 1. FIG.6 is a conceptual diagram illustrating C-FRAME based flow coordinationexecuted by the mobile apparatus 100 of FIG. 1.

Referring to FIGS. 1 to 6, the C-FRAME is the basic unit of dataprocessing and resource allocation. Collecting the C-FRAME andperforming subsequent operations can hardly be ceased in the middle, inorder to generate an accurate and timely context result. On the otherhand, it is often tolerable to temporarily pause the whole operationsonce the C-FRAME is completely processed.

Accordingly, ‘collecting and processing the C-FRAME’ is a unit ofresource allocation. Each C-FRAME is allocated with the right anddeserved amount of resources to collect and process the C-FRAME. A flowis coordinated by assigning a monitoring interval of the C-FRAME. Themonitoring interval of the C-FRAME may be set to guarantee theprocessing of the C-FRAME in each interval with the given resourceavailability.

In FIG. 5A, the CPU availability of the mobile apparatus is 40%. Alength of the C-FRAME is 2 sec. A CPU time to process the C-FRAME is 500ms. Thus, the maximum CPU usage of the C-FRAME is 500 ms/2 sec, 25%.

The CPU availability (40%) is greater than the maximum CPU usage (25%)of the C-FRAME so that the monitoring interval of the C-FRAME may be setto 2 sec. In FIG. 5A, the skip time when the sensing operator stopssensing the data does not exist. Thus, the sensing operator continuouslysenses the data.

In FIG. 5B, the CPU availability of the mobile apparatus is 10%. Alength of the C-FRAME is 2 sec. A CPU time to process the C-FRAME is 500ms. Thus, the maximum CPU usage of the C-FRAME is 500 ms/2 sec, 25%.

The CPU availability (10%) is less than the maximum CPU usage (25%) ofthe C-FRAME so that the C-FRAME may not use the maximum CPU usage butthe C-FRAME may use the CPU usage equal to or less than 10%. Therefore,the monitoring interval of the C-FRAME may be set to 5 sec. The CPUusage of the C-FRAME is 500 ms/5 sec, 10%.

In FIG. 5B, the sensing time of the sensing operator is 2 sec and theskip time when the sensing operator stops sensing the data is 3 sec.Thus, the sensing operator repeatedly senses the data and stops sensingthe data in a cycle of 5 sec.

For multiple concurrent flows, the coordination may be performed toassign different monitoring intervals of the C-FRAME for the differentflows. The flows may be coordinated to maximize the total utility ofcorresponding applications under the given resource availability.

In FIG. 6, when the CPU availability is 40%, the sensing stream of aflow f0 which has the C-FRAME having the length of 2 sec is sensed inthe monitoring interval of 2 sec. The CPU usage of the flow f0 is 500ms/2 sec, 25%. The sensing stream of a flow f1 which has the C-FRAMEhaving the length of 3 sec is sensed in the monitoring interval of 3sec. The CPU usage of the flow f0 is 200 ms/3 sec, about 6.67%. A sum ofthe CPU usage of the flow f1) and the CPU usage of the flow f1 is lessthan the CPU availability 40% so that the flow f1) and the flow f1maximally uses the CPU without the skip time.

When the CPU availability is changed to 10% from 40%, the CPU usages ofthe flow f0 and the flow f1 are adjusted. The CPU usages of the flow f0and the flow f1 may be determined using the utility function of the flowf0 and the utility function of the flow f1. The flow execution planner140 may determine the monitoring interval of the C-FRAME of the flow f0and the monitoring interval of the C-FRAME of the flow f1 to maximizethe utility of the flow f0 and the utility of the flow f1 under thegiven resource constraints. For example, the monitoring interval of theC-FRAME of the flow f0 may be set to about 7.1 sec. The skip time of theflow f0 may be about 5.1 sec. The CPU usage of the flow f0 is about 500ms/7.1 sec, about 7.0%. For example, the monitoring interval of theC-FRAME of the flow f1 may be set to about 6.7 sec. The skip time of theflow f1 may be about 4.7 sec. The CPU usage of the flow f1 is about 200ms/6.7 sec, about 3.0%.

As explained above, the actual execution of the flow in the mobileapparatus 100 is executed in the unit of the F-FRAME while theallocation is executed for each C-FRAME. Thus, the flexibility in thescheduling of the flow execution may be obtained. For example, the sizeof the single C-FRAME may be large (e.g. about 4 MB with IndoorNavi).Accordingly, processing the whole c-frame in a single step may causesudden CPU usage peaks and induce quite a long delay. According to thepresent exemplary embodiment, the flow is processed in the unit ofsmaller-size F-FRAMES so that the mobile apparatus 100 flexibly scheduleconcurrent applications in order to meet the delay requirements undercontentious situation.

FIG. 7 is a conceptual diagram illustrating an operation of the flowexecutor of FIG. 2.

Referring to FIGS. 1 to 7, the mobile system includes sensors such as anaccelerometer Accel, a gyroscope Gyro and a microphone Mic.

The sensing handler 160 includes a first sensing operator S01 operatinga sensing stream of the accelerometer Accel, a second sensing operatorS02 operating a sensing stream of the gyroscope Gyro and a third sensingoperator S03 operating a sensing stream of the microphone Mic.

The first to third sensing operators S01, S02 and S03 continuously sensethe data during sensing times corresponding to respective lengths of theC-FRAMEs and stop sensing during skip times which are determined bysubtracting the sensing times from the monitoring intervals. The firstto third sensing operators S01, S02 and S03 may have different sizes ofthe C-FRAMEs. The first to third sensing operators S01, S02 and S03 mayhave different monitoring intervals of the C-FRAMEs.

The sensing handler 160 manages the sensing operators each of which runsin a separate thread. The first sensing operator S01 stores accelerationsensing streams to a first F-FRAME queue in a unit of the F-FRAME. Thesecond sensing operator S02 stores gyro sensing streams to a secondF-FRAME queue in a unit of the F-FRAME. The third sensing operator S03stores sound sensing streams to a third F-FRAME queue in a unit of theF-FRAME. The first to third sensing operators S01, S02 and S03 may havedifferent sizes of the F-FRAME.

The processing handler 150 controls the execution of the subsequentprocessing operators. When the flow scheduler 130 picks one of theflows, the processing handler 150 takes the F-FRAME from the flow'sF-FRAME queue and processes the F-FRAME using the processing operator.For example, when the flow scheduler 130 selects a first flow, theprocessing handler 150 takes the acceleration F-FRAME from the firstF-FRAME queue and processes the acceleration F-FRAME using theprocessing operators P01 to P05. For example, when the flow scheduler130 selects a second flow, the processing handler 150 takes the gyroF-FRAME from the second F-FRAME queue and processes the gyro F-FRAMEusing the processing operators P06 to P08. For example, when the flowscheduler 130 selects a third flow, the processing handler 150 takes thesound F-FRAME from the third F-FRAME queue and processes the soundF-FRAME using the processing operators P09 to P14.

FIG. 8 is a conceptual diagram illustrating slack time estimation ofF-FRAME executed by the flow scheduler 130 of FIG. 2.

Referring to FIGS. 1 to 8, the flow scheduler 130 determines theexecution order of the F-FRAMEs piled up in the F-FRAME queues. Forexample, the flow scheduler 130 determines the execution order of theF-FRAMEs using following Formula 5.

maximizeΣsatisfy(ci)  [Formula 5]

ci means a context monitoring result for FLOWi generated by processingthe F-FRAMEs in the F-FRAME queue. The function of satisfy(ci) is abinary function to evaluate if ci is generated within a tolerable delay;satisfy(ci) is 1 if the delay of ci is less than the tolerable delay and0, otherwise. The tolerable delay is defined as the time taken togenerate the context monitoring result from the moment that the finalF-FRAME in the C-FRAME is ready.

The flow scheduler 130 may determine the execution order of the F-FRAMEsto obtain the maximum number of the context monitoring results byprocessing the F-FRAMEs in the tolerable delay.

In the present exemplary embodiment, the flow scheduler 130 may adopt aleast slack time method. The flow scheduler 130 calculates slack timesfor all F-FRAMEs in the F-FRAME queue. The flow scheduler 130 controlsthe processing handler 150 to process the F-FRAME having the least slacktime among the F-FRAMEs.

F-FRAMEi,j means j-th F-FRAME of i-th C-FRAME (C-FRAMEi). The slack timeof the F-FRAMEi,j may be calculated by subtracting the processing timeof remaining F-FRAMEs in the C-FRAMEi from a delivery deadline. Theslack time of the F-FRAMEi,j may be represented as following Formula 6.

slacktime(F-FRAMEi,j)=di+ri,j-epti,j  [Formula 6]

di is the tolerable delay of the FLOWi. ri,j is the remaining time tocollect the whole C-FRAME with upcoming F-FRAMEs. epti,j is the expectedtime to process the unprocessed F-FRAMEs in the C-FRAMEi. Theunprocessed F-FRAMEs are from F-FRAMEi,j to the final F-FRAME in theC-FRAMEi.

The flow scheduler 130 may ri,j using a position of the F-FRAMEi,j inthe C-FRAMEi and the number of the remaining F-FRAMEs and a samplingrate. The flow scheduler 130 continuously profiles processing times ofthe F-FRAMES and estimates epti,j from the profiling results for theF-FRAMEs of the previous C-FRAME.

According to the present inventive concept as explained above, thesensing operators collect the sensed data in the unit of the C-FRAME andthe processing operators executes the sensing flow in the unit of theF-FRAME so that resource usages of the concurrent applications may beadjusted. Thus the utility of the applications may be improved.

The flow execution planner 140 adjusts the monitoring interval of thesensing operators of the sensing handler 160 so that collecting thesensed data between the sensing operators may be properly controlled.

The flow scheduler 130 may properly adjust the execution order of theF-FRAMEs of the plurality of flows.

The foregoing is illustrative of the present inventive concept and isnot to be construed as limiting thereof. Although a few exampleembodiments of the present inventive concept have been described, thoseskilled in the art will readily appreciate that many modifications arepossible in the example embodiments without materially departing fromthe novel teachings and advantages of the present inventive concept.Accordingly, all such modifications are intended to be included withinthe scope of the present inventive concept as defined in the claims. Inthe claims, means-plus-function clauses are intended to cover thestructures described herein as performing the recited function and notonly structural equivalents but also equivalent structures. Therefore,it is to be understood that the foregoing is illustrative of the presentinventive concept and is not to be construed as limited to the specificexample embodiments disclosed, and that modifications to the disclosedexample embodiments, as well as other example embodiments, are intendedto be included within the scope of the appended claims. The presentinventive concept is defined by the following claims, with equivalentsof the claims to be included therein.

What is claimed is:
 1. A mobile apparatus comprising: a sensing handlerincluding a plurality of sensing operators, the sensing operator sensingdata during a sensing time corresponding to a size of C-FRAME andstopping sensing during a skip time, the C-FRAME being a sequence of thesensed data to produce a context monitoring result; and a processinghandler including a plurality of processing operators, the processingoperator executing the sensed data of the sensing operator in a unit ofF-FRAME, the F-FRAME being a sequence of the sensed data to execute afeature extraction operation.
 2. The mobile apparatus of claim 1,wherein the C-FRAME includes a plurality of the F-FRAMEs.
 3. The mobileapparatus of claim 1, further comprising a flow analyzer receivinginformation for a sensing flow from an application and determining asize of the C-FRAME of the sensing flow and a size of the F-FRAME of thesensing flow.
 4. The mobile apparatus of claim 3, wherein the flowanalyzer receives a necessary monitoring interval and a monitoring delayfrom the application, the necessary monitoring interval representing howoften the application needs to monitor a user's situation, themonitoring delay representing time taken to generate the contextmonitoring result from a moment that a final F-FRAME in the C-FRAME isready.
 5. The mobile apparatus of claim 4, further comprising a flowexecution planner determining a monitoring interval of the C-FRAME basedon the size of the C-FRAME, the size of the F-FRAME and the necessarymonitoring interval and outputting the monitoring interval of theC-FRAME to the sensing handler, wherein the monitoring interval of theC-FRAME of the sensing flow is substantially equal to a sum of thesensing time and the skip time.
 6. The mobile apparatus of claim 5,further comprising a resource monitor determining a CPU availability ofthe mobile apparatus and outputting the CPU availability to the flowexecution planner, wherein the flow execution planner determines themonitoring interval of the C-FRAME of the sensing flow based on the sizeof the C-FRAME, the size of the F-FRAME, the necessary monitoringinterval and the CPU availability.
 7. The mobile apparatus of claim 5,wherein the necessary monitoring interval has a type of utilityfunction, the utility function having utility values of the applicationaccording to the monitoring intervals.
 8. The mobile apparatus of claim7, wherein when the monitoring interval of a plurality of sensing flowsis pi and the utility function of the sensing flows is ui, the flowexecution planner determines the monitoring interval as a formula,maximizeΣui(pi).
 9. The mobile apparatus of claim 8, wherein when cti isa CPU time required to process the C-FRAME of a flow, FLOWi and 1−CPUfis a CPU availability for the applications, the flow execution plannerdetermines the monitoring interval under a constraint of Σcti/pi≦1−CPUf.10. The mobile apparatus of claim 8, wherein when eci is energy requiredto process the C-FRAME of a flow, FLOWi and Elimit is an energyavailability for the applications, the flow execution planner determinesthe monitoring interval under a constraint of Σeci/pi≦Elimit.
 11. Themobile apparatus of claim 5, wherein the sensing flows includerespective F-FRAME queues, further comprising a flow schedulerdetermining an execution order of the F-FRAMEs by selecting a F-FRAMEqueue among the F-FRAME queues.
 12. The mobile apparatus of claim 11,wherein the flow scheduler determines the execution order of theF-FRAMEs based on the monitoring delay.
 13. The mobile apparatus ofclaim 12, wherein a function of satisfy(ci) represents 1 if a contextmonitoring result for i-th flow, FLOWi is generated by processing theF-FRAMEs in the F-FRAME queue in the monitoring delay and represents 0,otherwise, and the flow scheduler determines the execution order of theF-FRAMEs using a formula, maximizeΣsatisfy(ci).
 14. The mobile apparatusof claim 13, wherein a j-th F-FRAME of an i-th C-FRAME (C-FRAMEi) isF-FRAMEi,j, di is a tolerable delay of the FLOWi, ri,j is a remainingtime to collect the remaining F-FRAMEs in the C-FRAMEi, epti,j is anexpected time to process the unprocessed F-FRAMEs in the C-FRAMEi and aslack time of the F-FRAMEi,j is determined as a formula,slacktime(F-FRAMEi,j)=di+ri, j-epti,j, and the flow scheduler selectsthe F-FRAME having a least slack time.
 15. The mobile apparatus of claim1, further comprising a CPU scheduler outputting a CPU quota to the flowexecution planner.
 16. A method of executing a sensing flow, the methodcomprising: sensing data during a sensing time corresponding to a sizeof C-FRAME and stopping sensing during a skip time using a plurality ofsensing operators, the C-FRAME being a sequence of the sensed data toproduce a context monitoring result; and executing the sensed data ofthe sensing operator in a unit of F-FRAME using a processing operator,the F-FRAME being a sequence of the sensed data to execute a featureextraction operation.
 17. The method of claim 16, wherein the C-FRAMEincludes a plurality of the F-FRAMEs.
 18. The method of claim 16,further comprising receiving information for a sensing flow from anapplication and determining a size of the C-FRAME of the sensing flowand a size of the F-FRAME of the sensing flow.
 19. The method of claim18, further comprising receiving a necessary monitoring interval and amonitoring delay from the application, the necessary monitoring intervalrepresenting how often the application needs to monitor a user'ssituation, the monitoring delay representing time taken to generate thecontext monitoring result from a moment that a final F-FRAME in theC-FRAME is ready.
 20. The method of claim 19, further comprisingdetermining a monitoring interval of the C-FRAME based on the size ofthe C-FRAME, the size of the F-FRAME and the necessary monitoringinterval, wherein the monitoring interval of the C-FRAME of the sensingflow is substantially equal to a sum of the sensing time and the skiptime.
 21. The method of claim 20, further comprising determining a CPUavailability of the mobile apparatus, wherein the monitoring interval ofthe C-FRAME of the sensing flow is determined based on the size of theC-FRAME, the size of the F-FRAME, the necessary monitoring interval andthe CPU availability.
 22. The method of claim 20, wherein the necessarymonitoring interval has a type of utility function, the utility functionhaving utility values of the application according to the monitoringintervals.
 23. The method of claim 20, wherein the sensing flows includerespective F-FRAME queues, further comprising determining an executionorder of the F-FRAMEs by selecting a F-FRAME queue among the F-FRAMEqueues.
 24. The method of claim 23, wherein the execution order of theF-FRAMEs is determined based on the monitoring delay.
 25. A method ofmonitoring a context, the method comprising: receiving a contextmonitoring request from an application; sensing data during a sensingtime corresponding to a size of C-FRAME and stopping sensing during askip time using a plurality of sensing operators based on the contextmonitoring request, the C-FRAME being a sequence of the sensed data toproduce a context monitoring result; executing the sensed data of thesensing operator in a unit of F-FRAME using a processing operator basedon the context monitoring request, the F-FRAME being a sequence of thesensed data to execute a feature extraction operation; and outputtingthe context monitoring result to the application.
 26. A contextmonitoring system comprising: a sensor generating sensed data; a mobileapparatus comprising: a sensing handler including a plurality of sensingoperators, the sensing operator sensing data during a sensing timecorresponding to a size of C-FRAME and stopping sensing during a skiptime, the C-FRAME being a sequence of the sensed data to produce acontext monitoring result; and a processing handler including aplurality of processing operators, the processing operator executing thesensed data of the sensing operator in a unit of F-FRAME, the F-FRAMEbeing a sequence of the sensed data to execute a feature extractionoperation; and an application receiving the context monitoring resultfrom the mobile apparatus.